A Parallel Architecture for High-Speed Data Compression

Data compression is becoming an essential component of high speed data communications and storage. Lossless data compression is when the decompressed data must be identical to the original. Textual substitution methods are among the most powerful approaches to lossless data compression, where repeated substrings are replaced by pointers into a dynamically changing dictionary of strings. We present a massively parallel architecture for textual substitution that is based on a systolic pipe of 3,839 identical processing elements that forms what is essentially an associative memory for strings that can “learn” new strings based on the text processed thus far. Key to the design of this architecture is the formulation of an inherently “top-down” serial learning strategy as a “bottom up” parallel strategy. A custom VLSI chip for this architecture that operates at 300 million bits per second has been fabricated.